06-n-'2004 02:35PM FROM-fiates i Cooper LLP +13106418798 T-673 P. 005 F-796 



INTHECLATM.<; 
The pending unamended claims are reproduced below. 

1. (ORIGINAL) A method for solving, in a computer, stochastic control problems of linear 
systems in high dimensions, comprising: 

(a) modeling, in die computet, a stmcmred Markov Decision Process (MDP), wherein a state 
space for the MDP is a polyhedron in a Euclidean space and one or more actions diiit are feasible in 
a state of the state space are linearly consttained with respect to the state; and 

(b) building, in the computer, one or more approcsdmations feom above and &om below to a 
value function for the state using representations diat facilitate die computation of approximately 
optimal actions at any given state by linear programming. 

2. (ORIGINAL) The method of claim 1, whetein die MDP comprises a supply chain 
planning process. 

3. (ORIGINAL) The method of claim 1, wherein the action space and the state space are 
continuous and related to each other through a system of linear constraints. 

4. (ORIGINAL) The method of claim 1, wherein die value function is convex and the 
mediod further comprises efadendy learning the value fbacrion in advance and representing die 
value fbnction in a way that allows for real-time choice of actions based thereon. 

5. (ORIGINAL) The method of claim 1, wherein the hnear function is approximated both 
from above and from below by piecewise linear and convex functions. 

6. (ORIGINAL) The mediod of claim 5, wherein the domains of linearity of the piecewise 
linear and convex functions are not stored expliddy. but rather are encoded dirongh a linear 
programming formulation. 

7. (ORIGINAL) The method of claim 5, wherein the domains of linearity of the piecewise 
linear and convex functions allow the functions to be optimized and updated in teal-time. 
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8. (ORIGINAl) The mechod of claim 1, wherein the value function can be (efficiently 
approximated both from above and from bdow. 

9. (ORIGINAL) The mediod of claim 1, wherein the approxtmations can be repeatedly 

refined. 

10. (ORIGINAL) The method of claim 1, wherein the value function can be efficiently 
approximated from above based on kno>xrledge of upper bounds on die function at each member of 
a selected set of states. 

11. (ORIGINAL) The method of claim 1, wherein tLc value function can be efficiently 
approximated from below baaed on linear functions that lie below the convex value fimctiorL 

12. (ORIGINAL) The method of claim 1, wherein liie value funcdon can be approximated 
successively. 

13. (ORIGINAL) A computerized apparatus for solving stochastic control problems of 
linear systems in high dimensions, comprising: 

(a) a computer; 

(b) logic, performed by the computer, for modeling a stmctored Madcov Decision Process 
(MDP), wherein a state space for the MDP is a polyhedron in a Euclidean space and one or more 
actions that are feasible in a state of the stare space are linearly congttained wiuh respect to the state; 
and 

(c) logic, performed by the computer, for building one or more approximations from above 
and from below to a value frmctioa for the stare using representations that facilitate the computation 
of approximately optimal actions at any given state by linear ptogcamming. 

14. (ORIGINAL) The apparatus of daim 13, wherein the MDP comprises a supply chain 
planning process. 

15. (ORIGINAL) The apparatus of claim 13, wherein the acdon space and die state space 
are continuous and related to each other through a system of linear constraints. 
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16. (ORIGINAL) The apparams of claim 13, wherein the value function is convex and the 
logic further comprises efficiently le armng thi* value fimctioii in advance and represendbig the value 
function in a way that allows for real-tttne choice of actions based thereon. 

17. (ORIGINAI,) The apparatus of claim 13, wherein the linear function is approximated 
both &otn above and fi:om beJow by piecewise linear and convex functions. 

18. (ORIGINAL) The apparatus of claim 17, wherein the domains of linearity of the 
piecewise linear and convex functions are not stored explicitly, but rather arc encoded through a 
linear programming formulation. 

19. (ORIGINAL) The apparatus of claim 17, wherein the domains of linearity of the 
piecewise linear and convex funcdons allow the functions to be optimiaed and updated in real-mne. 

20. (ORIGINAL) The apparatus of claim 13, wherein the value function can be efficiently 
approximated both fcom above and from below. 

21. (ORIGINAL) The apparatus of claim 13, whexeia the approxitnations can be repeatedly 

re£ned. 

22. (ORIGINAL) The apparatus of claim 13» wherein the value function can be cffidendy 
approximated fcom above based on knowledge of upper bounds on the function at each member of 
a selected set of states. 

23. (ORIGINAL) The apparatus of claim 13, wherein the value function can be efficiently 
approximated from below based on linear functions diat lie below the convex value function. 

24. (ORIGINAL) Tlie apparatus of claim 13, wherein the value function can be 
approximated successively. 
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25. (ORIGINAL) An article of manufacture embodying logic for solving stochastic control 
problems of linear systems in high dinutnsions, the logic comprising; 

(a) modeling a structured Markov Decision Process (MDP), wherein a stare space for the 
MDP is a polyhedron in a Euclidean space and one or more actions that ate feasible in a state of the 
state space arc linearly consttained with respect to ihc state; and 

(b) building one or mote approximations feom above and fi:om below to a value funcrion for 
the state using representations that fiidlitate the computation of approximately optimal actions at 
any given state by linear programming. 

26. (ORIGINAL) The article of manufacmrc of claim 25, Nst^herein die MDP compiises a 
supply chain planning process. 

27. (OKJGINAL) The article of manufacture of claim 25, wherein the action space and the 
state space arc continuous and related to each other through a system of linear constraints. 

28. (ORIGINAL) The article of manufacture of claim 25, wherein the value function is 
convex and the logic further comprises efficiendy learning the value function in advance and 
representing the value function in a way that allows for real-time choice of actions based thereon. 

29. (ORIGINAL) The article of manufactore of claim 25, wherein the linear fimction is 
approximated both from above and from below by piecewise linear and convex functions. 

30. (ORIGINAL) The article of manufiicftitti of claim 29, wherein the domains of linearity 
of the piccewise linear and convex fiinctions are not stored escplidtly. but rather axe encoded 
through a linear progtamming formulation. 

31. (ORIGINAL) The article of manufacture of claim 29, wherein the domains of linearity 
of the piecewise linear and convex funcbons allow the functions to be optimized and updared in 
real-time. 



32. (ORIGINAL) The article of manufacture of claim 25, wherein the value function can be 
effidendy approximated both firom above and from below. 
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33. (ORIGINAL) Tbc article of manufacture of claitu 25, wherein the approxunations can 
be repeatedly refined. 

34. (ORIGINAL) The axdcle of mflnufacrute of claim 25, wherein the value function can be 
effidenriy approximated from above based on knowledge of upper bounds on the function at each 
member of a selected set of states. 

35. (ORIGINAL) The article of manufacture of daim 25, wherein the valxie function can be 
efficiently approximated firom below based on linear functions that he below the convex value 
AmctioQ. 

36. (ORIGINAL) Hie article of manufactuie of claim 25, wherein the value function can be 
approximatied successively. 
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